Robust Ridge Regression for High-Dimensional Data

Ridge regression, being based on the minimization of a quadratic loss function, is sensitive to outliers. Current proposals for robust ridge regression estimators are sensitive to “bad leverage observations”, cannot be employed when the number of predictors p is larger than the number of observations n; and have a low robustness when the ratio p=n is large. In this paper a ridge regression estimate based on Yohai’s (1987) repeated M estimation (“MM estimation”) is proposed. It is a penalized regression MM estimator, in which the quadratic loss is replaced by an average of (ri=b ), where ri are the residuals and b the residual scale from an initial estimator, which is a penalized S estimator; and is a bounded function. The MM estimator can be computed for p > n and is robust for large p=n: A fast algorithm is proposed. The advantages of the proposed approach over its competitors are demonstrated through both simulated and real data.